Robotic planning in real-world scenarios typically requires joint optimization of logic and continuous variables. A core challenge to combine the strengths of logic planners and continuous solvers is the design of an efficient interface that informs the logical search about continuous infeasibilities. In this paper we present a novel iterative algorithm that connects logic planning with nonlinear optimization through a bidirectional interface, achieved by the detection of minimal subsets of nonlinear constraints that are infeasible. The algorithm continuously builds a database of graphs that represent (in)feasible subsets of continuous variables and constraints, and encodes this knowledge in the logical description. As a foundation for this algorithm, we introduce Planning with Nonlinear Transition Constraints (PNTC), a novel planning formulation that clarifies the exact assumptions our algorithm requires and can be applied to model Task and Motion Planning (TAMP) efficiently. Our experimental results show that our framework significantly outperforms alternative optimization-based approaches for TAMP.

For applications that require processing large amounts of text at inference time, Large Language Models (LLMs) are handicapped by their limited context windows, which are typically 2048 tokens. In-context learning, an emergent phenomenon in LLMs in sizes above a certain parameter threshold, constitutes one significant example because it can only leverage training examples that fit into the context window. Existing efforts to address the context window limitation involve training specialized architectures, which tend to be smaller than the sizes in which in-context learning manifests due to the memory footprint of processing long texts. We present Parallel Context Windows (PCW), a method that alleviates the context window restriction for any off-the-shelf LLM without further training. The key to the approach is to carve a long context into chunks (``windows'') that fit within the architecture, restrict the attention mechanism to apply only within each window, and re-use the positional embeddings among the windows. We test the PCW approach on in-context learning with models that range in size between 750 million and 178 billion parameters, and show substantial improvements for tasks with diverse input and output spaces. Our results motivate further investigation of Parallel Context Windows as a method for applying off-the-shelf LLMs in other settings that require long text sequences.

Monocular Depth Estimation (MDE) is a fundamental problem in computer vision with numerous applications. Recently, LIDAR-supervised methods have achieved remarkable per-pixel depth accuracy in outdoor scenes. However, significant errors are typically found in the proximity of depth discontinuities, i.e., depth edges, which often hinder the performance of depth-dependent applications that are sensitive to such inaccuracies, e.g., novel view synthesis and augmented reality. Since direct supervision for the location of depth edges is typically unavailable in sparse LIDAR-based scenes, encouraging the MDE model to produce correct depth edges is not straightforward. In this work we propose to learn to detect the location of depth edges from densely-supervised synthetic data, and use it to generate supervision for the depth edges in the MDE training. %Despite the 'domain gap' between synthetic and real data, we show that depth edges that are estimated directly are significantly more accurate than the ones that emerge indirectly from the MDE training. To quantitatively evaluate our approach, and due to the lack of depth edges ground truth in LIDAR-based scenes, we manually annotated subsets of the KITTI and the DDAD datasets with depth edges ground truth. We demonstrate significant gains in the accuracy of the depth edges with comparable per-pixel depth accuracy on several challenging datasets.

We introduce MuJoCo MPC (MJPC), an open-source, interactive application and software framework for real-time predictive control, based on MuJoCo physics. MJPC allows the user to easily author and solve complex robotics tasks, and currently supports three shooting-based planners: derivative-based iLQG and Gradient Descent, and a simple derivative-free method we call Predictive Sampling. Predictive Sampling was designed as an elementary baseline, mostly for its pedagogical value, but turned out to be surprisingly competitive with the more established algorithms. This work does not present algorithmic advances, and instead, prioritises performant algorithms, simple code, and accessibility of model-based methods via intuitive and interactive software. MJPC is available at: github.com/deepmind/mujoco_mpc, a video summary can be viewed at: dpmd.ai/mjpc.

结肠镜检查是一种常规门诊手术，用于检查结肠和直肠的任何异常，包括息肉，憩室和结肠结构的狭窄。临床医生的大量时间用于在结肠镜检查过程中拍摄的快照，以维持医疗记录或进一步研究。自动化此步骤可以节省时间并提高流程的效率。在我们的工作中，我们收集了一个由专家注释的过程中的120个结肠镜检查视频和2416张快照的数据集。此外，我们开发了一种基于新颖的，视觉转化器的地标检测算法，该算法可以从结肠镜检查过程中鉴定出关键的解剖标志（阑尾孔，回肠瓣膜/盲肠地标和直肠翻新）。我们的算法在预处理过程中使用自适应伽马校正，以保持所有图像的一致亮度。然后，我们将视觉变压器用作特征提取主链和完全连接的基于网络的分类器头，将给定的框架分为四个类：三个地标或非地标框架。我们将视觉变压器（VIT-B/16）主链与RESNET-101和Convnext-B骨干进行了比较，这些骨干和Convnext-B骨干也接受了类似训练。我们报告了快照的测试数据集上的视觉变压器主链的精度为82％。

在视频分析中，背景模型具有许多应用，例如背景/前景分离，变更检测，异常检测，跟踪等。但是，尽管在静态相机捕获的视频中学习这种模型是一项公认的任务，但在移动相机背景模型（MCBM）的情况下，由于算法和可伸缩性挑战，成功率更加重要。由于相机运动而产生。因此，现有的MCBM在其范围和受支持的摄像头类型的限制中受到限制。这些障碍还阻碍了基于深度学习（DL）的端到端解决方案的这项无监督的任务。此外，现有的MCBM通常会在典型的大型全景图像或以在线方式的域名上建模背景。不幸的是，前者造成了几个问题，包括可扩展性差，而后者则阻止了对摄像机重新审视场景先前看到部分的案例的识别和利用。本文提出了一种称为DEEPMCBM的新方法，该方法消除了上述所有问题并实现最新结果。具体而言，首先，我们确定与一般和DL设置的视频帧联合对齐相关的困难。接下来，我们提出了一种新的联合一致性策略，使我们可以使用具有正则化的空间变压器网，也不是任何形式的专业化（且不差异）的初始化。再加上在不破坏的稳健中央矩（从关节对齐中获得）的自动编码器，这产生了一个无端到端的无端正规化MCBM，该MCBM支持广泛的摄像机运动并优雅地缩放。我们在各种视频上展示了DEEPMCBM的实用程序，包括超出其他方法范围的视频。我们的代码可在https://github.com/bgu-cs-vil/deepmcbm上找到。

任何稀疏编码方法的最终目标是从几个嘈杂的线性测量值（一个未知的稀疏向量）中准确恢复。不幸的是，这个估计问题通常是NP-HARD，因此始终采用近似方法（例如Lasso或正交匹配的追踪）来接近它，从而使准确性以较小的计算复杂性进行了交易。在本文中，我们为稀疏编码开发了一种量子启发的算法，前提是，与经典近似方法相比，量子计算机和ISING机器的出现可能会导致更准确的估计。为此，我们将最一般的稀疏编码问题作为二次不受约束的二进制优化（QUBO）任务提出，可以使用量子技术有效地最小化。为了在旋转数量（空间复杂性）方面也有效地得出QUBO模型，我们将分析分为三种不同的情况。这些由表达基础稀疏向量所需的位数来定义：二进制，2位和一般的定点表示。我们使用有关Lightsolver量子启发的数字平台的模拟数据进行数值实验，以验证我们的QUBO公式的正确性，并证明其优于基线方法的优势。

最近有很多不可能的结果表明，在与对抗对手的马尔可夫游戏中最小化的遗憾在统计学上和计算上是棘手的。然而，这些结果都没有排除在所有各方采用相同学习程序的假设下，遗憾最小化的可能性。在这项工作中，我们介绍了第一种（据我们所知）在通用马尔可夫游戏中学习的算法，该算法在所有代理商执行时提供了sublinear后悔保证。我们获得的边界是为了置换遗憾，因此，在此过程中，意味着融合了相关的平衡。我们的算法是分散的，计算上有效的，并且不需要代理之间的任何通信。我们的主要观察结果是，在马尔可夫游戏中通过策略优化的在线学习基本上减少了一种加权遗憾的最小化形式，而未知权重由代理商的策略顺序的路径长度确定。因此，控制路径长度会导致加权的遗憾目标，以提供足够的适应性算法提供统一的后悔保证。

为了推动满足所有人需求并使医疗保健民主化的健康创新，有必要评估各种分配转变的深度学习（DL）算法的概括性能，以确保这些算法具有强大的态度。据我们所知，这项回顾性研究是第一个开发和评估从跨种族，年龄和性别的长期跳动间隔的AF事件检测的深度学习模型（DL）模型的概括性能（DL）模型的概括。新的复发DL模型（表示为ARNET2）是在2,147名患者的大型回顾性数据集中开发的，总计51,386小时连续心电图（ECG）。对来自四个中心（美国，以色列，日本和中国）的手动注释测试集评估了模型的概括，总计402名患者。该模型在以色列海法的Rambam医院Holter Clinic的1,730个Consecutives Holter记录中进一步验证了该模型。该模型的表现优于最先进的模型，并且在种族，年龄和性别之间进行了广泛的良好。女性的表现高于男性和年轻人（不到60岁），并且在种族之间显示出一些差异。解释这些变化的主要发现是心房颤动患病率更高（AFL）的群体的性能受损。我们关于跨组的ARNET2相对性能的发现可能对选择相对于感兴趣群的首选AF检查方法具有临床意义。

深神经网络（DNN）的成功在很大程度上取决于计算资源。虽然DNN经常在云服务器上使用，但在边缘设备上运行DNN的需求越来越大。边缘设备的计算资源通常受到限制，但是，通常将多个边缘设备部署在相同的环境中，并且可以可靠地相互通信。在这项工作中，我们建议通过允许多个用户在推理过程中协作以提高其准确性来促进DNN在优势上的应用。我们的机制（创造的机制）基于每个设备的各种预测因子，在推理过程中构成了模型集合。为了减轻通信开销，用户共享量化的功能，我们提出了一种将多个决策汇总到单个推论规则中的方法。我们分析了边缘合奏所引起的延迟，表明其性能提高是以在通信网络上的共同假设下的较小延迟成本为代价的。我们的实验表明，配备紧凑型DNN的Edge合奏的协作推断显着提高了让每个用户在本地推断出的精度，并且可以使用大于整体中所有网络的单个集中式DNN胜过。